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ABSTRACT 



This paper addresses issues related to the possible 
inclusion of items in the Schools and Staffing Survey (SASS) to measure 
instructional practices and teaching effectiveness. The SASS is a national 
survey of schools and teachers conducted by the National Center for Education 
Statistics. This report explores the purposes that can be served by measuring 
instructional practices on a national scale, and then examines how teaching 
effectiveness is conceptualized. Another section describes some strengths and 
limitations of observational studies of classroom instruction and makes a 
similar assessment of survey studies. Some attention is given to curricular 
reforms and how their impact might be assessed. The last sections of the 
paper suggest specific ways in which instructional practices and curricular 
content might be measured through the SASS, including the selection of 
specific school subjects and grade levels for attention. A number of large 
survey studies have already provided valuable information about curriculum 
and instruction in U.S. classrooms, and the SASS could join them. There does 
not appear to be any federal program in which instructional practices and 
opportunities to learn will be monitored in the future and the SASS, with its 
large sample of teachers, seems to be an excellent vehicle for the 
measurement of both curriculum and instructional practices. The SASS also 
seems ideally suited to monitor the classroom consequences of reforms such as 
curriculum standards. (Contains 50 references.) (SLD) 
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Should SASS Measure Instructional Processes and Teacher 

Effectiveness? 1 

Susan S. Stodolsky 
University of Chicago 
December, 1995 Draft 

This paper addresses issues related to the possible 
inclusion of items in SASS to measure instructional practices and 
teaching effectiveness. In order to answer the question posed, 
the paper explores what purposes can be served by measuring 
instructional practices on a national scale. It then examines 
how teaching effectiveness is conceptualized. Turning next to 
substantive and methodological concerns, a section describes some 
strengths and limitations of observational studies of classroom 
instruction and makes a similar assessment of survey studies. 

Some attention is then given to curricular reforms and how their 
impact might be assessed. The last sections of the paper suggest 
specific ways in which instructional practices and curricular 
content might be measured through SASS, including the selection 
of specific school subjects and grade levels for attention. 



Why collect national data on instruction? 

We begin by briefly examining some of the main reasons to 
collect information about instructional practices/processes on a 
national scale. If we are to understand, monitor, and improve our 
nation's schools, accurate and timely empirical, descriptive data 
about how schools work must be available. The activities that 



’This paper was written at the request of Policy Study 
Associates for use by NCES. The author is solely responsible for 
the ideas presented. 

I would like to acknowledge the generous assistance of John 
Mullens at PSA in providing numerous documents and surveys. 

Andrew Porter and Fred Newmann, both at the University of 
Wisconsin, Madison, Leland Cogan of the TIMSS at Michigan state 
University, Joan Talbert at CRC at Stanford University, and 
Valerie Lee at University of Michigan, also made surveys or 
papers available. 

I have tried to cite sources as appropriate throughout the 
paper . However , the surveys and papers that form the core used in 
preparing this paper are much like an extended family. Surveys 
have uncharted historical connections to one another, with items 
borrowed and adapted freely. Many commonalities in thinking 
appear in papers on the topic. I apologize in advance for any 
omissions in citations or for mistakenly citing a source that is 
not the definitive one. 
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take place in classrooms to engender student learning and 
development are the heart of any school's educational efforts. It 
is in the transactions between and among teachers, students, 
materials and tasks that deliberate efforts to educate occur. 

Descriptive information about how teaching and learning 
occur in classrooms and about what is taught provides the basis 
for monitoring the status of instruction in a large number of 
settings. Such information might provide periodic assessments of 
stability and change in instruction, particularly as changes 
relate to deliberate efforts to reform or alter curriculum and 
instruction. Similarly, if collected along with knowledge of 
particular policy initiatives, curriculum standards, or changes 
in teacher preparation or staff development, information 
describing classroom instruction might help track the impact of 
various policies on what transpires in classrooms. 

Descriptive information about classroom processes also can 
contribute to the deliberations of teachers, teacher educators, 
subject matter and other educational associations, and policy 
makers at local, state, and national levels. Basic researchers 
also benefit from information about what actually goes on in 
classrooms. 

Many argue that classroom process information is most 
valuable when connected to student achievement and attainment 
data. Data from NELS have been analyzed recently by Kupermintz, 
Ennis, Hamilton, Talbert and Snow (1995) and Lee & Smith (1995). 
Both research groups found significant relationships between 
certain measures of instructional practices (e.g. emphasis on 
higher order thinking), teacher attitudes (e.g. willingness to 
alter instructional practices if students are not learning) and 
student performance on both math knowledge (lower mental process) 
and math reasoning (higher mental process) items. A similar 
analysis of teachers' responses to the CLAS survey by Wiley & 

Koon (1995) also demonstrates the potential for connections 
between instructional items and student attainment. These 
studies support the importance of monitoring instructional 
practices in the nation. 

SASS as currently structured is not linked to student data 
on achievement or other outcomes and a considerable redesign and 
change in sampling strategy would be required to do so. However, 
links to achievement can be direct and empirical as when teachers 
and their students are studied, or links can be putative and 
conceptual based on known or assumed connections between 
practices and achievement. Existing research and theory can be 
used to formulate the presumptive connections and might also 
inform the design of empirical studies. 
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How is teaching effectiveness conceptualized? 

The charge for this paper includes a consideration of 
whether measures of instructional practices and teaching 
effectiveness should be included in future SASS instruments. It 
must be noted here that a broad consensus on a definition of 
effective or good teaching does not exist. Empirical evidence, 
theory and values along with specified criteria for effectiveness 
all enter into a conception of effective teaching. In addition, 
considerable evidence that instructional practices need to be 
tailored to subject matter, developmental levels of students, and 
other factors is now available, suggesting that effectiveness 
comes in a number of varieties. 



Although not all teachers and policy makers endorse one view 
of effective teaching in a given time period, visions of 
effective teaching change over time. For instance, during the 
late 1970's and 1980' s, the process-product research program 
(Brophy & Good, 1986) assumed there were generic characteristics 
of good teaching (i. e. they apply to all school subjects and 
grade levels considered) . By focusing primarily on features of 
teacher-centered instruction, this influential, empirical 
research program identified a number of teacher behaviors (direct 
instruction model) that correlated with student gains on 
standardized achievement tests in reading and math, primarily in 
elementary schools. 

Critiques of the process-product view point out that the 
correlational method embodied the existential fallacy (Stodolsky, 
1988) . That is, only currently used practices could enter the 
model of effective teaching. The use of data on individual 
teacher behaviors decontextualized instruction and made it 
difficult to know how to put the instructional program into 
operation as a combination of student and teacher behaviors (an 
intact lesson structure). The model excluded behaviors that might 
be subject- or grade-level specific. The model did not examine 
student behaviors. The model adopted a transmission view of 
teaching. Effectiveness was operationalized by achievement tests 
that almost exclusively contained lower-mental process skill 
items . 

Now, as evident in many standards and reform documents, a 
constructivist point of view of learning and teaching is holding 
sway. This view directs attention to students' active role in the 
^■®**^ning process. Classroom arrangements such as group work, 
debate and discussion are believed integral to effective 
instruction. In addition, teaching and learning are assumed to be 
different from one school subject (or even topic) to another. No 
fully general model of effective teaching and learning is 
expected. Last, different student criteria are employed to judge 
effectiveness. More emphasis is placed on reasoning, problem 
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solving, creative production and long-term products. Methods of 
student assessment beyond standardized achievement tests are 
endorsed. 

The transmission and constructivist views of teaching both 
may have a proper place in the analysis of teaching 
effectiveness. Flexibility in instructional strategies may be a 
hallmark of effective teaching. Different instructional practices 
may be desirable depending on instructional goals and lesson 
formats. Instruments to assess effective practices must contain 
an appropriate range of items to tap lesson structures, content, 
instructional strategies and teacher and student activities. 

There is a danger in only assessing popular visions of effective 
teaching which may not be widely implemented or universally 
appropriate at any given time. 

Some Features of classroom Activity and Teacher Behavior: 
Observational Studies 

Observations of instructional activity are often thought to 
be the most valid method of data collection. While observational 
studies are generally beyond the scope of proposed SASS 
activities because they are very expensive, direct observation 
can often be used in early stages of instrument development to 
provide relevant categories and items for surveys and other 
measurement approaches. Observations also have a place in 
validity studies. 

Observational studies provide accumulated knowledge from 
which to formulate productive questions. Useful reviews of 
research on curriculum and teaching are provided by Shulman 
(1986) and Dar 1 ing -Hammond & Snyder (1992). Here we take a 
selective look at past observational research on teacher behavior 
and classroom activity. 

Observational studies (e.g. Good and Brophy, 1986; Goodlad, 
1984) have documented a robust picture of teacher-centered 
instruction primarily oriented toward lower-level cognitive 
goals. Recitations, variants on lecture, and seatwork are the 
primary instructional formats used in most classrooms. However, 
systematic variation occurs when subject matter is examined. 
Similarity of instructional practices across teachers may be 
greater in subjects such as mathematics, than in subjects such as 
social studies or English (Stodolsky, 1988). Variation is also 
tied to teachers' conceptions of subject matter and goals 
(Grossman & Stodolsky, 1994, 1995; Shulman, 1987). 

A variety of contextual and situational factors produce 
variation in teaching and consequently limit the stability and 
generalizability that can be expected in studies of teacher 
behavior, especially at the level of the individual teacher. As 
noted elsewhere (Stodolsky, 1990) subject matter, grade level, 
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lesson type, and lesson goal account for variation in teaching 
behaviors and instructional arrangements. In addition, the type 
of students and track level of courses (Oakes, 1985) along with 
district policies, type of school, and other institutional 
factors may all influence a teacher's choice of curricular 
content and instructional methods. 

Limitations of data from observational studies 

Perhaps one of the most important limitations of available 
observational studies, a feature shared with survey studies, is 
that the contexts studied are limited. The preponderance of 
large-scale studies have been conducted with elementary school 
teachers of reading and math. A few have focused on social 
studies instruction at the elementary and high school levels 
(Stodolsky , 1988; Newmann, 1992; Newmann & Wehlage, 1995). While 
there are a variety of small-scale observational studies, 
including studies of classroom discourse, our knowledge is not 
deep with respect to the state of classroom curriculum and 
instruction in fields such as science, social studies, English, 
foreign language or the arts. We also have surprisingly few 
observations of what actually takes place in high school 
classrooms. 

Another limitation of available observational research is 
that it has focused primarily on teacher behaviors. A more 
ecological approach to classroom settings, such as employed by 
Doyle (1983), Gump (1982) and Stodolsky (1988), examines 
classroom activities, and incorporates knowledge of what both 
students and teachers do during instruction, along with knowledge 
of materials and tasks. However, studies of classroom ecology 
have been relatively rare. In most observational studies, when 
students are observed it is to assess their on-task behavior or 
involvement. 

Nevertheless, observations can provide evidence of real 
instructional experiences unrivaled by other methods. 

Par ti c ularly if one wants to understand the qualities of 
transactions that occur in classrooms and their intellectual and 
social features, observations can play a possibly unique role. 
Observations, done properly, can reveal the connections between 
what is taught and how it is taught — observations can preserve 
classroom events as they occur together. The issue for NCES or 
others striving for a national picture of curriculum and 
instruction is under what circumstances, if any, direct 
°b s er va tio n should be used as a data gathering approach. 
Similarly, it is important to determine how useful information 
obtained with other methods such as surveys or teacher logs can 
be for the same purpose. 
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Studies of Curriculum and Instructional Processes: 

Survey Research 

A number of large survey studies, often funded by NCES, NSF, 
and OERI, have provided valuable information about curriculum and 
instruction in our nation's classrooms. The main contributors to 
our knowledge about curriculum and instruction on a national 
scale are NELS:88, NAEP, Reform Up close (RUC) and SIMS. Weiss 
(1993) provides some useful information on instructional 
practices among math and science teachers. The validity of using 
surveys as a measurement tool in the area of curriculum and 
instruction has also been examined (Burstein et al,1995; Porter, 
1995; TIMSS) and survey development is ongoing (Porter, 1995) . 

Useful reviews and analysis of many of these survey projects 
can be found in NCES working papers (Leighton, 1994a, b; Leighton & 
Mullens, 1994; Mullens et al, 1994, 1995), a report by Porter 
(1995) and the work of Schmidt & McKnight (1995). A catalogue of 
instruments measuring the enacted curriculum in math and science 
at the middle and high school levels is now available (Porter & 
Smithson, 1995) . A list of sources for major surveys can be found 
in Appendix 1. 

The surveys distinguish between plans (intended curriculum 
objectives) and actions (enacted or implemented curriculum) . 
Most of these surveys emphasize curriculum and instruction in 
action, although goals for instruction may also be measured. To 
varying degrees, these surveys seek to measure plans 
(instructional goals), to document what is taught (content/ topics 
and intellectual processes, time allocations, emphases) ; how 
instruction is organized (pedagogy, teacher and student activity, 
homework and tests) ; and resource use (e.g. technology, 
textbooks) . The surveys are often described as measures of 
students' opportunity to learn (OTL) , a term borrowed from the 
IEA studies. McDonnell (1995) provides a useful discussion of the 
OTL construct. 

The uneven coverage of contexts found in observational 
studies is also characteristic of the survey research. In an 
interesting juxtaposition, however, most of the surveys deal with 
high school or 8th grade instruction while the observation 
studies are mainly at the elementary level. In fact, with the 
exception of NAEP 4th grade surveys, Weiss (1994), the Consortium 
on Chicago School Reform (1994) which borrows from NELS and RUC, 
and the CRC (1994) survey of elementary math teachers in 
California, it was difficult to locate surveys of the enacted 
curriculum given on a large scale at the elementary level. High 
school coverage is also somewhat uneven. NAEP targets 12th grade 
and therefore obtains information primarily about advanced 
courses. There is reason to believe that the practices used in 
more advanced courses may differ to some extent from those in the 
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earlier years of high school. Burstein et al (1995) document 
that teachers of more advanced courses are more accurate in 
reporting topic/content coverage and emphases. 

The surveys are also uneven with respect to subject matter 
coverage. Largely due to the efforts of NSF, major survey 
development has occurred in math and considerable attention has 
also been paid to science instruction. Applebee (1981; 1992) 
conducted national surveys on the teaching of writing and 
literature at the high school level which provide modest amounts 
of information about instructional practices; the Applebee work 
might be a starting point for further survey development in 
English along with available NELS items on English. According to 
Andy Porter, the CPRE School-Based Management Survey (SBM) also 
contains items dealing with instruction in language arts and 
social studies at the elementary and high school levels. The 
items follow the four-part scheme developed by Porter and others 
to assess teachers' objectives, content covered, modes of 
instruction and cognitive processes. The content items in the 
CPRE surveys are rather general and might provide only a starting 
point for item development in English and social studies. With 
the exceptions mentioned plus NAEP items and some history items 
in NELS, significant recent efforts to develop surveys of 
enacted curriculum and instructional practices have not been 
undertaken at a level of effort similar to that in math and 
science in the fields of English, social sciences, foreign 
language, and other subjects including the arts. In order to 
obtain information about instructional practices in a range of 
subjects and grade levels, considerable new survey development is 
required. 

It seems more than financial support has led to so much 
attention to curriculum and instruction measures in math. 
Mathematics lends itself to a systematic analysis of its content, 
topics, and operations because it is the best defined and 
probably least contentious of all school subjects. Compared to 
other subjects, there is considerable agreement among math 
teachers and teacher educators about best practice. 

Mapping science curriculum topics may pose a greater 
challenge than mapping topics in math. Our own work (Stodolsky 
and Grossman, 1995) on five academic subjects and an analysis of 
English by Grossman (1993) and Elbow (1990), suggest that there 
is less agreement about content and teaching methods in subjects 
such as English and social studies. Many versions of these 
subjects exist and teachers of English and social studies expect 
considerable autonomy in the selection of course content, 
especially because they are not constrained by a perceived 
content sequence. (Science teachers also report freedom in choice 
of topics to teach, but share a commitment to the scientific 
method.) The difficulties confronted in developing social studies 
and language arts curriculum standards, confirm the lack of 
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consensus in 'these fields. The development of surveys with 
detailed topical analyses for English and social studies, such as 
those available in math and science, presents a challenge in 
curricular analysis and instrument development not yet addressed. 

Curricular Reforms 

Before moving on to specific suggestions regarding future 
directions for SASS, the issue of curricular reforms needs some 
discussion. Studies of teachers in settings in which reforms are 
underway have found a mixed picture at best. For example, in case 
studies of mathematics teachers attempting to implement the 
California Math Frameworks, Cohen and Peterson (1990) found only 
modest changes from conventional practice, confirming the 
suggestion by Burstein et al (1995) that new practices are 
"layered" on to old ones. On the other hand, certain changes in 
math teaching such as the introduction of calculators seem more 
widespread (Weiss, 1994). 

Research on the Coalition for Essential Schools (Muncey & 
McQuillan, 1993; Little, 1995) documents great variety in the 
extent to which teachers adopt Coalition principles. However, 
many teachers report using process writing approaches such as 
those advocated by the National Writing Project (Freedman, 1987; 
NAEP Report Card on Writing) . 

To help understand the implementation of reforms, a first 
step might be to learn what teachers actually know about proposed 
reforms and standards. Adequate teacher knowledge and 
understanding of reforms is far from guaranteed just because 
standards are published or new frameworks drawn up. The CRC 
survey provides excellent examples of items that could be used to 
assess teacher knowledge of curriculum frameworks. It would also 
be desirable to obtain knowledge of organizational support and 
provision of resources for reform in departments and schools. In 
addition, it is important to determine if teachers are asked to 
act simultaneously on a number of policy initiatives which may 
not be consistent with one another. 

If one of the purposes in monitoring instruction in the 
nation is to provide information about the progress of curricular 
reforms, it must be assured that the item pool used to measure 
curriculum and instruction is adequately tailored to the reforms 
advocated in each subject matter studied. 

An examination of the standards for curriculum in science 
(NRC, 1994; Rutherford & Ahlgren, 1990), social studies 
(NCSS, 1995) and mathematics (NCTM, 1989;1991) suggests different 
degrees of emphasis on changing pedagogy and changing content. 

The math standards may be most explicit with respect to the 
vision they embody of pedagogy consistent with the recommended 



8 




15 



standards. 2 The use of open-ended and student-generated problems 
and investigations which take place over a number of days are 
examples of a constructivist pedagogy endorsed by NCTM. Specific 
items have been written to address features of pedagogy in the 
NCTM standards; the CRC survey has some excellent examples. 

Porter (1995) reports making use of the NCTM standards and NSTA 
standards in developing opportunity to learn topic items for math 
and science. 



Specialized terminology or language poses a possible problem 
in instrument development with items geared toward reforms. 
Burstein et al. (1995) in their validity study of surveys in math 
found that teachers did not always interpret terms in the same 
manner (e.g "math modeling" had a number of different meanings to 
the teachers they studied). A term like "investigations" used in 
the NCTM and California math frameworks might carry a variety of 
connotations to respondents. Indeed, the term "reform" itself is 
not used equivalently by those reporting about it. 

Last, in some cases the new standards are predicated on 
teacher mastery of subject matter and pedagogical content 
knowledge not currently widely held in the teaching force. The 
TIMSS survey and the CRC survey for math teachers, include items 
to reveal teachers' conceptual understanding of mathematical 
material along with pedagogy. It seems likely that items of this 
type would predict student attainment, and help us document 
barriers to implementation of reforms. As such, they seem 
important to include in any effort to measure curricular reform. 

Should 8A88 include measures of instructional practice? 

Except for NAEP, there does not appear to be any federal 
program in which instructional practices and opportunity to learn 
will be monitored in future. SASS, with its large sample of 
teachers, seems an excellent vehicle for the measurement of 
curriculum and instructional practices. However, the inclusion of 
a fairly comprehensive set of items on content (e.g. as in TIMSS 
or the Porter OTL four— dimension scheme) would involve a lot of 
additional respondent time. Further, to adequately monitor 
pedagogy and track reforms, additional items would be needed. 

Since NAEP is an ongoing program that taps into curriculum 
and instruction in a number of school subjects (although maybe 
not very deeply) , an optimal plan for SASS would complement and 
supplement efforts planned under NAEP. Some school subjects and 
grade levels not regularly covered by NAEP should be included in 
SASS. At the same time, more targeted efforts to link with NAEP 



Of course the NCTM standards also recommend change in what 
is taught in math classes. 
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and/or assist NAEP to enhance its curriculum and instruction 
measures would be highly desirable. Some links to NAEP would also 
provide tie-ins to student survey responses about their 
instructional experiences. 

SASS seems ideally suited to monitor the classroom 
consequences of reforms such as curriculum standards. (It may be 
asking too much to monitor the myriad of other reforms underway.) 
A selection of specific school subjects and grade levels seems 
the best strategy here as well. However, in order to maximize 
insight into how reforms work, it would be desirable to have more 
teacher respondents from a given school than has been the case in 
P^®^ious SASS sampling, so that information about the presence of 
Particular reform efforts in the schools could be obtained. 
Linking with NAEP under selected circumstances would also benefit 
from more clustering of teachers in schools. 

Item selection 

Let us begin by examining measures of content taught. As 
discussed earlier, there is a substantial pool of items to use in 
measuring the content taught in science and mathematics, 
especially at the middle school and high school levels. Limited 
topical analysis is also available in U. S. and world history, 
although not the broader social studies. English is essentially 
unmapped as is much of the elementary school curriculum. 
(Exceptions are the three-dimensional content structure developed 
by Freeman, Porter and others for fourth-grade mathematics and 
some items from NAEP dealing with reading and writing 
instruction) . The four— dimension topic items such as developed by 
Porter (1995) for his recent OTL study for math and science, seem 
a suitable model for item sets to be used in SASS. The four 
dimensions include two dimensions of topics and the degree of 
emphasis each receives, cognitive activities (with time 
distribution) and the medium (mode) of instruction (with time 
distribution) . For school subjects other than math and science, 
item development analogous to the Porter model would have to be 
undertaken. 

There are quite a few items and item types dealing with 
pedagogy or teaching methods that seem applicable to most 
subjects and grade levels, although a careful analysis would be 
required to assure that practices found in elementary school 
classrooms were adequately sampled. The language in which methods 
are described might also require modification and field testing 
when applied in contexts other than those surveyed to date. In 
addition, specialized language from reform documents should be 
used with caution and fully pilot tested to assure common 
understandings . 

Burstein et al (1995) recommend against the inclusion of 
items measuring goals, as they did not find a good match with 
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responses and other data sources such as the goals inferred in 
tests or teacher assignments. On the other hand, they did find 
meaningful relationships between endorsement of reform goals and 
reform practices, but not between endorsement of traditional 
goals and traditional practices. This issue would seem to require 
further study before eliminating goal items from national 
surveys. The data pattern suggests in part that most teachers 
believe traditional goals are worthwhile, even those who are 
moving their practice in the direction of reform. This finding 
seems another example of the tendency of teachers to add on to 
their practice without giving up old patterns. Thus, some 
tensions inherent in change may be revealed effectively through 
analysis of goal items. Although not the highest priority, if 
respondent time allows, goal items should be retained. 

As Leighton (1994a) notes, subject— specif ic questionnaires 
have been the rule recently. Many common instructional items 
reappear in surveys for teachers of different subjects in 
addition to specific items for each subject. The 1994-95 SASS 
Follow-up Teacher Questionnaire has a number of sections dealing 
with teaching methods that are promising and which build on 
development work from other surveys we have discussed. While a 
good starting point, a careful review would have to be made for 
appropriateness to grade levels and school subjects selected for 
study. Also, there may be some overlap in constructs if four- 
dimension content items such as those in Porter are also in the 
survey . 

In addition to content/ topic items and pedagogy items, new 
items should be developed that assess teachers' specific 
knowledge of reforms. The CRC survey provides some good examples 
of such items for the California Math Frameworks. Teacher's 
subject matter and pedagogical content knowledge required for 
implementing reform should also be measured. 3 A particularly 
promising item format has been used in TIMSS and the CRC survey, 
among others. The items ask teachers to envision an 
instructional sequence of lesson parts used to teach a specified 
topic. For example, the CRC survey asked questions about 
instruction dealing with fractions in an open-response format. 

The TIipiS items are more structured (See examples in Appendix 2) . 
These items tap lesson organization, content emphasis, 
pedagogical content knowledge and subject matter knowledge and 
may be an effective way to tie together features of instructional 
processes and content in a manner that approximates what actually 
happens in classrooms. It seems worth considering if such items 
could be developed for use in subjects other than math. 



Resource use, especially what textbooks and other 
materials are used is beyond the scope of this paper. The 
omission does not reflect a lack of importance. 



In creating item sets for a survey, it would be desirable to 
include measures of teacher efficacy and willingness to adapt 
instruction as these scales have important predictive power in 
connection with other instructional items. Professional 
development activities and participation in subject area and 
other networks should also be assessed. Such measures would 
reflect teachers' opportunities to learn about or deepen 
understanding of new approaches. These scales could be part of a 
teacher background section of the survey. 

To create respondent time for the suggested content /pedagogy 
and teacher knowledge items, we suggest two strategies. One is 
to eliminate certain parts of the current SASS survey since items 
have been administered over many years and may be given to a 
subsample or less frequently. In particular, items dealing with 
teacher control over policies such as discipline, hiring of new 
teachers could be omitted. The list of perceived problems 
(poverty, tardiness, etc) might also be eliminated or given to a 
subsample of teachers. 

The second strategy takes us into the realm of sampling. It 
seems that not all items need to be given to all teachers. We 
will recommend selecting teachers of certain grades and school 
subjects to respond to the curriculum and instruction survey. 
Other respondents could be used to answer more general questions 
from SASS. In addition, even teachers within the recommended 
grades and subjects could be directed (say, by use of their 
birthday as a sorting mechanism) to answer only certain parts of 
the survey. 

Who should be surveyed? 

As suggested, it may not be appropriate for all SASS 
teachers to respond to items dealing with curriculum and 
instructional practices. Targeting teachers of certain subjects 
and grade levels would seem a good approach. However, the 
decision regarding what school subjects and grades to select is 
not an easy one. 

Because so much investment has been made in instrument 
development in math, and because the NCTM standards were in the 
vanguard, it seems appropriate to use math as one of the target 
subjects. The scope of surveys about math should be expanded to 
include math in the upper elementary grades (4-6) along with 
middle school and high school. If costs permit, science is 
another well developed area that could be studied in the middle 
and high school levels. In addition to math. Porter (1991) 
suggests English as an important understudied area. Leighton 
(1994a) suggests history as another possibility. At the middle 
and high school levels, both are plausible options, with history 
having somewhat of an edge in terms of existing instrumentation. 

If history was selected as a focal subject, the elementary grades 
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should again be included. Fifth-grade is typically the year when 
U. S. history appears in the elementary curriculum. So surveys in 
grades 4-6 would make sense. However, the elementary social 
studies curriculum is quite diverse and content items should 
range well beyond history to articulate with actual practice. 

At the elementary level, time allocations to subject differ 
snd this information might form one criteria for school subject 
selection. We know that reading (language arts) , then math are 
given the most instructional time in the elementary grades, 
especially at the primary level. In fact, little time may be 
devoted to science instruction in many elementary schools while 
modest amounts are allocated to social studies. By the upper 
elementary grades, social studies may be on a slightly more even 
footing with language arts and math. 

Within social studies, there is contention about the 
direction the subject should take. Both history and social 
studies standards (NCHS, 1994; NCSS, 1995) have been formulated 
and there is considerable tension among adherents to each set of 
standards. A SASS survey aimed at charting reform in this area, 
would be challenged to accommodate differing points of view. On 
the other hand, the English/ language arts have the problem that 
their curriculum standards are not completed, in fact, it appears 
that work on the English /language arts standards has stopped, at 
least for the time being. Defining English /Language Arts at the 
elementary level is also messy. In some cases students will 
receive instruction primarily geared to developing skills in 
reading and/or writing; in other instances the instructional 
program will be more directed toward literature. Based on these 
factors, the choice of history/social studies might make the most 
sense. 

A cautionary note should be sounded with regard to the 
subject-specific focus of this discussion. Current instruments 
and our discussion have assumed that instruction is 
compartmentalized by subject. Empirical evidence suggests this is 
still largely true, but a number of curricular reforms call for 
more subject matter integration and interdisciplinary teaching. 

It seems important to bear this in mind in reviewing items for 
inclusion in SASS and in thinking about how to select teachers. 
One hopes that there are teachers whose instructional programs 
are strongly integrated for whom answering a more conventional 
survey could be problematic. Such teachers may be more likely to 
be found in elementary schools, but high school programs 
emphasizing subject integration are also being implemented. 

Support for some instrument development studies seems in 
order as an important step to prepare for the next SASS cycle. 
These studies would be directed at enhancing our capability to 
measure curriculum and instruction in subjects hitherto 
understudied — especially elementary social studies, and math, and 



history/social studies at the middle and high school level. Work 
might also begin on mapping the English/ language arts curriculum 
for future studies. The instrument development studies would 
involve multi-method investigations that could determine the 
validity of pilot items and other methods. Effective use of 
teacher logs, collections of teacher assignments, exams and other 
materials; textbook analyses, and classroom observations might be 
incorporated in the instrument development process. We agree 
with Burstein et al (1995) that validation studies should 
regularly accompany the introduction of new surveys. 

Additional small studies conducted through SASS (perhaps in 
Follow-Up surveys) might delve into topics that might be of 
interest to the nation from time to time. Illustrative is the 
section of the SASS 1994-95 Teacher Follow-up Questionnaire which 
inquires about portfolio assessments. Inquiry into special topics 
such as this could be a regular part of SASS, with only a 
fraction of teacher respondents being asked to provide 
information. In this manner, not all teachers would take exactly 
the same set of items, but reliable information could still be 
obtained on a number of interesting issues. 
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2. Many students have trouble relating ratios to fractions when they are asked 
to relate part of a set of objects to the whole set For example, when asked 
There are 2 boys in a class for every 3 girls in the class. What fraction of the 
students are boys?" Many students would answer 2/3 rather than 2/5. If 
you were working with a class in which many students had this kind of • 
misunderstanding, what approach or sequence of approaches do you 
believe would best help students learn? 

Place a Tin the box in the right-hand margin next to the approach you believe to 
be the best. If you believe other approaches would also be acceptable, place a 
number in the box next to each one indicating the order in which you would 
consider using it. You need not choose more than one approach. Leave blank the 
box for any approach you do not consider acceptable. 



a) I would review with my students the section of the textbook that explains this 

concept. 

b) I would make the situation more concrete by having the students help me make up 

the class roster for a class with two boys and three girls. From this class roster, I 
would then ask the students to work on finding a solution to the problem. 

c) I would ask several students to explain their thinking about this problem and ask 
other students to comment on what seems helpful and not helpful with these 
explanations. If this did not clear up the difference in understandings, I would at 
least better understand my students' thinking and could choose activities to provide 
them with experiences that might lead them to the more conventional, useful idea. 

d) I would present several situations of this sort and after getting students to answer 

what fraction of the class were boys and what fraction girls, I would ask the 
students to use calculators to find what percent of the class were boys and what 
percent girls. Then I would ask them if the percentages of boys and girls in 
class added to 100 percent. 

e) I would discuss which sets of objects that were involved in the 
situation with a diagram as the one shown at right and that the 
fraction needed is 

-- -777 m which is not ec ’ ual t0 ~ - I A, B disjoint 
n (A U B) n(B ) 

Many students do not even realize that the set A U B is involved as well as set A 
and set B 



OO O © O 

... 0 0 GOO 

f) I would relate this kind of situation to the general idea of ratio as o o o © © 

represented by discrete objects such as 2:3 which is. represented, o o © © © 

for example, by thediagram alright. Then I would investigate with °° ooe 

students all the various fractions that could be made in such a situation 

g) Which of the approaches do you believe to be the least acceptable 
approach? 

Place the letter of that approach in the box 



ALIO (etttt) 




THANK YOU for the thought, time, and effort you have put into completing this 

questionnaire. 



BEST COPY AVAILABLE 
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13. How would you begin a unit on fractions with 



your class? What would you do first? 



What would you do next? 



14. 



Which of the following could be used to illustrate what 3/4 means? (CIRCLE ALL THAT APPLY.) 



a. 

b. 

c. 

d. 




/ 



/ 



/ o o o 



Stand four children up in front of the room and 
place hats on three of them 

None of these because . 



2 

3 

4 



15. 



S,° c f the Rowing story problems could be used to illustrate what 
(UHCLt ALL THAT APPLY.) 



1 1/4 divided by 1/2 means? 



a. 

b. 

c. 

d. 

e. 



You want to split 1 1/4 pies evenly between two families. How much should 
each family get? 

youend up wkh? and ^ S °° n d ° Uble y ° Ur money - H ° W much money would 

You are making some homemade taffy and the recipe calls for 1 1/4 cups of 
butter. How many sticks of butter (each stick = 1/2 cup) will you need?„. 

I'm not sure 

None of these. Instead: 



1 

2 

3 

4 

5 
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